How to manipulate a gene list

The golden rule before we start Always keep a backup of the output from the pipeline, never work on the original data!

There are various tasks you might wish to perform on a gene list

  • Search for our favourite gene
  • Sort / Rank according to statistic or p-value
  • Filter to obtain genes with particular cut-off and fold-change
  • Re-order / re-arrange columns

Our recommended tools to perform such operations would be a programming language such as R / Python so that the operations can be scripted and automated. As a compromise, our Galaxy server provides various operations through an intuitive interface and allows the user to build a workflow to a chain of simple operations.

Different online tools or GUIs will take different types of input, but will probably involve some combination of the operations mentioned above. Therefore, we will introduce some tools in Galaxy that should allow you to manipulate your data into the required form. Later, we will give examples of workflows for particular gene set enrichment / pathways analysis tools.

In the first generic example, we will show how to filter a gene list using Galaxy:-

How to apply an alternative filter to the gene list in Galaxy

Upload the gene list in csv form

The gene list .csv file can be uploaded into Galaxy. In this example we want to choose the file with no p-value cut-offs applied

Convert to tabular

Before we can go further, Galaxy needs to convert the data you just uploaded into tabular form by replacing commas with a tab. This can be done using the Text Manipulation -> Convert delimiters to TAB option. Make sure that Commas is selected from the drop-down.

Filter

We can apply filtering using the menu option Filter and Sort -> Filter data on any column using simple expressions. Here we use the condition c7 <0.01 and c3 > 1.5 to require that Column 7 (adjusted p-value) is less than 0.05 and Column 3 (log fold-change) is greater than 1.5.

You could also use this menu to filter on genes on a particular chromosome or within a certain range of start and end positions

Selecting columns

Sorting by a column

We can also sort the table by values in a particular column. e.g. the log\(_2\) fold change.

  • can also sort by multiple columns
    • e.g. Chromosome ## R workflow

For those keen on R, equivalent operations can be performed with the dplyr package. See our intermediate R course for details.

library(dplyr)
deTable <- read.csv("t47d_Treatment_DEA_Prog-vs-Control_all.csv")
filteredTable <- filter(deTable, padj < 0.05, log2FoldChange > 1.5)
sortedTable <- arrange(filteredTable, log2FoldChange)
reducedTable <- select(filteredTable, X,baseMean,log2FoldChange)

Gene-Ontologies and Pathways

Gene-Ontology Analysis

In the early days of microarray analysis, people were happy if they got a handful of differentially-expressed genes that they could validate or follow-up. However, with later technologies (and depending on the experimental setup) we might have thousands of statistically-significant results, which no-one has the time to follow-up. Also, we might be interested in pathways / mechanisms that are altered and not just individual genes.

In this section we move towards discovering if our results are biologically significant. Are the genes that we have picked statistical flukes, or are there some commonalities.

There are two different approaches one might use, and we will cover the theory behind both

Theory Part I: Over-representation analysis

  • “Threshold-based”: require defintion of a statistical threshold to define list of genes to test (e.g. FDR < 0.01)
  • Hypergeometric test or Fisher’s Exact test generally used.

The question we are asking here is;

“Are the number of DE genes associated with Theme X significantly greater than what we might expect by chance alone?”

Where Theme X could be genes belonging to a particular GO (Gene Onotology) term or pathway.

Let’s imagine that we have a bag full of balls. Each balls represents a gene in the gene universe. - Paint the balls representing our selected list grey, and paint the rest red.

In this small example, we can know in advance the total number of balls and total number in our category of interest

  • Total number of balls: 40
  • Total number of interesting (grey) balls: 10

Now, lets select a number (say, 12) of the balls at random without seeing into the bag and look at what we get

We can express a particular choice of balls as a table:-

In Selection Not In Selection Total
Grey Selected 8 2 10
Grey Not Selected 4 26 30
Total 12 6 40

The formula for Fishers exact test is;

\[ p = \frac{\binom{a + b}{a}\binom{c +d}{c}}{\binom{n}{a +c}} = \frac{(a+b)!(c+d)!(a+c)!(b+d)!}{a!b!c!d!n!} \] with :-

In Selection Not In Selection Total
Grey Selected a b a + b
Grey Not Selected c d c + d
Total a + c b +d a + b + c + d (=n)

or less formally;

P = (ways of choosing grey balls) X (ways of non-grey balls amongst subset) / ways of choosing subset

Fortunately, we have software to calculate these quantities!

Software for conducting a over-representation test (goseq)

We will be using goseq, which is a software package available through Bioconductor. However, rather having to write R code, we will be using the package through our institute’s Galaxy server.

This package has been specifically-developed for use with RNA-seq data. Plenty of methods have been applied to microarray data, but the assumptions might not hold for RNA-seq. For instance, goseq will adjust for the length of a gene.

Preparing the data for an over-representation test

We can start with the list that contains results for all genes; Let’s take t47d_Treatment_DEA_Prog-vs-Control_all.csv

deTable <- read.csv("t47d_Treatment_DEA_Prog-vs-Control_all.csv")
deTable

The goal is to obtain a table with two columns; the first containing the gene identifier, and the second being TRUE or FALSE indicating whether the gene is differentially-expressed at a given cut-off (e.g. 0.05).

There are several ways of doing this. If you wish to see the R code (using dplyr), click the ‘Code’ button on the right.

library(dplyr)
deTable <- read.csv("t47d_Treatment_DEA_Prog-vs-Control_all.csv")
mutate(deTable, DE = padj < 0.05) %>% 
  mutate(DE = ifelse(is.na(DE),FALSE,DE)) %>% 
  select(X, DE) -> filteredTable
filteredTable
  write.table(filteredTable,"de-table-for-goseq.txt",quote=FALSE,row.names=FALSE)

Manipulating the file can also be done in Galaxy.

Importing the Gene list into Galaxy

Import the csv file

Get Data -> Upload Data

Convert to tabular form

Text Manipulation -> Convert delimiters to TAB

Select Commas from drop-down menu

Remove header line

Text Manipulation -> Select last lines from a dataset

Make note of how many lines in file, N and remove N-1 last lines with this tool

Retain columns of interest

Put c1,c2,c3,c4,c5,c6,c7,c12 to retain columns 1,2,3,4,5,6,7 and 12

Text Manipulation -> Cut columns from a table

The cleaned dataset

The output from the previous step will be referred to as our cleaned dataset

Performing a Gene Set Analysis in Galaxy

From our cleaned table in the previous steps, we need to compute if each gene is DE at 0.05 significance level

Adding an extra column

Text Manipulation -> Compute an expression on every row

Use the expression c7<0.05 to test if the adjusted p-value is less than 0.05

Which should give something like this

Extracting the columns needed for goseq

Text Manipulation -> Cut columns from a table Choose c1,c9 as the columns to cut

Which should give the following

Running goseq

*NGS:RNA-seq -> goseq

Select Genome version hg19 in this case. Could also specify which type of identifiers can be found in the first column. Ensembl ID is correct in this case.

Theory Part II: Threshold-free

For these tests, we don’t have to specify a statistical threshold and use the test statistics from all genes as the input to the test. The popular Gene Set Enrichment Analysis (GSEA) uses this approach. These tests can be used to detect differential expression for a group of genes, even when the effects are too small or there is too little data to detect the genes individually.

Subramanian et al, PNAS 2005

Subramanian et al, PNAS 2005

  • Rank all your genes according to test statistic or fold-change
  • For a particular gene set of interest, look down the gene list and compute a score
    • Go up one unit on the y-axis if the gene you encounter is contained in your gene list
    • Go down one unit if the gene is not contained in the gene list
    • Plot the score against position in gene list
    • If there is a peak at the left or right, then your set is enriched.

The Broad institute provides a version of GSEA that can be run via a java application. The required input text contains a set of gene identifiers that are ordered according to some test statistic.

library(dplyr)
deTable <- read.csv("deseq2_results/t47d_Treatment_DEA_Prog-vs-Control_all.csv")
deTable %>% arrange(desc(stat)) %>% 
  dplyr:::select(symbol, stat) %>% 
  filter(!is.na(symbol)) -> orderedTable
orderedTable

Preparing the gene set for a GSEA analysis

Ranking the genes

From the cleaned table created above, sort on column 5 (the test statistic)

Filter and Sort -> Sort data in ascending or descending order

Giving the output:-

Cutting the columns

Extract columns 8 (gene symbol) followed by test statistic; c8,c5

Text Manipulation -> Cut columns from a table

Running the GSEAPreranked tool

In our Galaxy server we provide an interface tofgsea, which is a Bioconductor package for performing the GSEA algorithm on a pre-ranked gene list.

We can give it the table we have generated in the previous step

Gives all results ranked according to..

  • Can we identify gene sets that are significant at 0.05?
  • What genes are over- and under-represented?
  • Can you run the tool on another dataset?
LS0tCnRpdGxlOiAiQmV5b25kIHRoZSBHZW5lIExpc3QiCmF1dGhvcjogIk1hcmsgRHVubmluZzsgbWFyayAnZG90JyBkdW5uaW5nICdhdCcgY3J1ay5jYW0uYWMudWsiCmRhdGU6ICdgciBmb3JtYXQoU3lzLnRpbWUoKSwgIkxhc3QgbW9kaWZpZWQ6ICVkICViICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogeWVzCiAgICB0b2NfZmxvYXQ6IHllcwogIGh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHllcwotLS0KCmBgYHtyIHNldHVwLCBpbmNsdWRlPUZBTFNFLGVjaG89RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gRkFMU0UpCmBgYAoKIyBIb3cgdG8gbWFuaXB1bGF0ZSBhIGdlbmUgbGlzdAoKVGhlIGdvbGRlbiBydWxlIGJlZm9yZSB3ZSBzdGFydCAqKipBbHdheXMga2VlcCBhIGJhY2t1cCBvZiB0aGUgb3V0cHV0IGZyb20gdGhlIHBpcGVsaW5lLCBuZXZlciB3b3JrIG9uIHRoZSBvcmlnaW5hbCBkYXRhKioqIQoKVGhlcmUgYXJlIHZhcmlvdXMgdGFza3MgeW91IG1pZ2h0IHdpc2ggdG8gcGVyZm9ybSBvbiBhIGdlbmUgbGlzdAoKLSBTZWFyY2ggZm9yIG91ciBmYXZvdXJpdGUgZ2VuZQotIFNvcnQgLyBSYW5rIGFjY29yZGluZyB0byBzdGF0aXN0aWMgb3IgcC12YWx1ZQotIEZpbHRlciB0byBvYnRhaW4gZ2VuZXMgd2l0aCBwYXJ0aWN1bGFyIGN1dC1vZmYgYW5kIGZvbGQtY2hhbmdlCi0gUmUtb3JkZXIgLyByZS1hcnJhbmdlIGNvbHVtbnMKCk91ciByZWNvbW1lbmRlZCB0b29scyB0byBwZXJmb3JtIHN1Y2ggb3BlcmF0aW9ucyB3b3VsZCBiZSBhIHByb2dyYW1taW5nIGxhbmd1YWdlIHN1Y2ggYXMgUiAvIFB5dGhvbiBzbyB0aGF0IHRoZSBvcGVyYXRpb25zIGNhbiBiZSBzY3JpcHRlZCBhbmQgYXV0b21hdGVkLiBBcyBhIGNvbXByb21pc2UsIG91ciBHYWxheHkgc2VydmVyIHByb3ZpZGVzIHZhcmlvdXMgb3BlcmF0aW9ucyB0aHJvdWdoIGFuIGludHVpdGl2ZSBpbnRlcmZhY2UgYW5kIGFsbG93cyB0aGUgdXNlciB0byBidWlsZCBhIHdvcmtmbG93IHRvIGEgY2hhaW4gb2Ygc2ltcGxlIG9wZXJhdGlvbnMuCgpEaWZmZXJlbnQgb25saW5lIHRvb2xzIG9yIEdVSXMgd2lsbCB0YWtlIGRpZmZlcmVudCB0eXBlcyBvZiBpbnB1dCwgYnV0IHdpbGwgcHJvYmFibHkgaW52b2x2ZSBzb21lIGNvbWJpbmF0aW9uIG9mIHRoZSBvcGVyYXRpb25zIG1lbnRpb25lZCBhYm92ZS4gVGhlcmVmb3JlLCB3ZSB3aWxsIGludHJvZHVjZSBzb21lIHRvb2xzIGluIEdhbGF4eSB0aGF0IHNob3VsZCBhbGxvdyB5b3UgdG8gbWFuaXB1bGF0ZSB5b3VyIGRhdGEgaW50byB0aGUgcmVxdWlyZWQgZm9ybS4gTGF0ZXIsIHdlIHdpbGwgZ2l2ZSBleGFtcGxlcyBvZiB3b3JrZmxvd3MgZm9yIHBhcnRpY3VsYXIgZ2VuZSBzZXQgZW5yaWNobWVudCAvIHBhdGh3YXlzIGFuYWx5c2lzIHRvb2xzLgoKSW4gdGhlIGZpcnN0IGdlbmVyaWMgZXhhbXBsZSwgd2Ugd2lsbCBzaG93IGhvdyB0byBmaWx0ZXIgYSBnZW5lIGxpc3QgdXNpbmcgR2FsYXh5Oi0KCiMjIEhvdyB0byBhcHBseSBhbiBhbHRlcm5hdGl2ZSBmaWx0ZXIgdG8gdGhlIGdlbmUgbGlzdCBpbiBHYWxheHkKCiMjIyBVcGxvYWQgdGhlIGdlbmUgbGlzdCBpbiBjc3YgZm9ybQoKVGhlIGdlbmUgbGlzdCBgLmNzdmAgZmlsZSBjYW4gYmUgdXBsb2FkZWQgaW50byBHYWxheHkuIEluIHRoaXMgZXhhbXBsZSB3ZSB3YW50IHRvIGNob29zZSB0aGUgZmlsZSB3aXRoIG5vIHAtdmFsdWUgY3V0LW9mZnMgYXBwbGllZAoKIVtdKGltYWdlcy9maWx0ZXIxLnBuZykKCiMjIyBDb252ZXJ0IHRvIHRhYnVsYXIKCkJlZm9yZSB3ZSBjYW4gZ28gZnVydGhlciwgR2FsYXh5IG5lZWRzIHRvIGNvbnZlcnQgdGhlIGRhdGEgeW91IGp1c3QgdXBsb2FkZWQgaW50byAqdGFidWxhciogZm9ybSBieSByZXBsYWNpbmcgY29tbWFzIHdpdGggYSB0YWIuIFRoaXMgY2FuIGJlIGRvbmUgdXNpbmcgdGhlICoqKlRleHQgTWFuaXB1bGF0aW9uKioqIC0+ICoqKkNvbnZlcnQgZGVsaW1pdGVycyB0byBUQUIqKiogb3B0aW9uLiBNYWtlIHN1cmUgdGhhdCAqQ29tbWFzKiBpcyBzZWxlY3RlZCBmcm9tIHRoZSBkcm9wLWRvd24uCgohW10oaW1hZ2VzL2ZpbHRlcjIucG5nKQoKIyMjIEZpbHRlcgoKV2UgY2FuIGFwcGx5IGZpbHRlcmluZyB1c2luZyB0aGUgbWVudSBvcHRpb24gKioqRmlsdGVyIGFuZCBTb3J0KioqIC0+ICoqKkZpbHRlciBkYXRhIG9uIGFueSBjb2x1bW4gdXNpbmcgc2ltcGxlIGV4cHJlc3Npb25zKioqLiBIZXJlIHdlIHVzZSB0aGUgY29uZGl0aW9uIGBjNyA8MC4wMSBhbmQgYzMgPiAxLjVgIHRvIHJlcXVpcmUgdGhhdCBDb2x1bW4gNyAoYWRqdXN0ZWQgcC12YWx1ZSkgaXMgbGVzcyB0aGFuIDAuMDUgKmFuZCogQ29sdW1uIDMgKGxvZyBmb2xkLWNoYW5nZSkgaXMgZ3JlYXRlciB0aGFuIDEuNS4KCgohW10oaW1hZ2VzL2ZpbHRlcjMucG5nKQoKWW91IGNvdWxkIGFsc28gdXNlIHRoaXMgbWVudSB0byBmaWx0ZXIgb24gZ2VuZXMgb24gYSBwYXJ0aWN1bGFyIGNocm9tb3NvbWUgb3Igd2l0aGluIGEgY2VydGFpbiByYW5nZSBvZiBzdGFydCBhbmQgZW5kIHBvc2l0aW9ucwoKIyMgU2VsZWN0aW5nIGNvbHVtbnMKCiFbXShpbWFnZXMvZmlsdGVyNC5wbmcpCgojIyBTb3J0aW5nIGJ5IGEgY29sdW1uCgpXZSBjYW4gYWxzbyBzb3J0IHRoZSB0YWJsZSBieSB2YWx1ZXMgaW4gYSBwYXJ0aWN1bGFyIGNvbHVtbi4gZS5nLiB0aGUgbG9nJF8yJCBmb2xkIGNoYW5nZS4KIVtdKGltYWdlcy9maWx0ZXI1LnBuZykKCi0gY2FuIGFsc28gc29ydCBieSBtdWx0aXBsZSBjb2x1bW5zCiAgICArIGUuZy4gQ2hyb21vc29tZSAKIyMgUiB3b3JrZmxvdwoKRm9yIHRob3NlIGtlZW4gb24gUiwgZXF1aXZhbGVudCBvcGVyYXRpb25zIGNhbiBiZSBwZXJmb3JtZWQgd2l0aCB0aGUgYGRwbHlyYCBwYWNrYWdlLiBTZWUgb3VyIFtpbnRlcm1lZGlhdGUgUl0oaHR0cDovL2Jpb2luZm9ybWF0aWNzLWNvcmUtc2hhcmVkLXRyYWluaW5nLmdpdGh1Yi5pby9yLWludGVybWVkaWF0ZS8pIGNvdXJzZSBmb3IgZGV0YWlscy4gCgpgYGB7ciBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KGRwbHlyKQpkZVRhYmxlIDwtIHJlYWQuY3N2KCJ0NDdkX1RyZWF0bWVudF9ERUFfUHJvZy12cy1Db250cm9sX2FsbC5jc3YiKQpmaWx0ZXJlZFRhYmxlIDwtIGZpbHRlcihkZVRhYmxlLCBwYWRqIDwgMC4wNSwgbG9nMkZvbGRDaGFuZ2UgPiAxLjUpCnNvcnRlZFRhYmxlIDwtIGFycmFuZ2UoZmlsdGVyZWRUYWJsZSwgbG9nMkZvbGRDaGFuZ2UpCnJlZHVjZWRUYWJsZSA8LSBzZWxlY3QoZmlsdGVyZWRUYWJsZSwgWCxiYXNlTWVhbixsb2cyRm9sZENoYW5nZSkKYGBgCgojIEdlbmUtT250b2xvZ2llcyBhbmQgUGF0aHdheXMKCiMjIEdlbmUtT250b2xvZ3kgQW5hbHlzaXMKCkluIHRoZSBlYXJseSBkYXlzIG9mIG1pY3JvYXJyYXkgYW5hbHlzaXMsIHBlb3BsZSB3ZXJlIGhhcHB5IGlmIHRoZXkgZ290IGEgaGFuZGZ1bCBvZiBkaWZmZXJlbnRpYWxseS1leHByZXNzZWQgZ2VuZXMgdGhhdCB0aGV5IGNvdWxkIHZhbGlkYXRlIG9yIGZvbGxvdy11cC4gSG93ZXZlciwgd2l0aCBsYXRlciB0ZWNobm9sb2dpZXMgKGFuZCBkZXBlbmRpbmcgb24gdGhlIGV4cGVyaW1lbnRhbCBzZXR1cCkgd2UgbWlnaHQgaGF2ZSB0aG91c2FuZHMgb2Ygc3RhdGlzdGljYWxseS1zaWduaWZpY2FudCByZXN1bHRzLCB3aGljaCBuby1vbmUgaGFzIHRoZSB0aW1lIHRvIGZvbGxvdy11cC4gQWxzbywgd2UgbWlnaHQgYmUgaW50ZXJlc3RlZCBpbiBwYXRod2F5cyAvIG1lY2hhbmlzbXMgdGhhdCBhcmUgYWx0ZXJlZCBhbmQgbm90IGp1c3QgaW5kaXZpZHVhbCBnZW5lcy4KCkluIHRoaXMgc2VjdGlvbiB3ZSBtb3ZlIHRvd2FyZHMgZGlzY292ZXJpbmcgaWYgb3VyIHJlc3VsdHMgYXJlICoqKmJpb2xvZ2ljYWxseSBzaWduaWZpY2FudCoqKi4gQXJlIHRoZSBnZW5lcyB0aGF0IHdlIGhhdmUgcGlja2VkIHN0YXRpc3RpY2FsIGZsdWtlcywgb3IgYXJlIHRoZXJlIHNvbWUgY29tbW9uYWxpdGllcy4gCgpUaGVyZSBhcmUgdHdvIGRpZmZlcmVudCBhcHByb2FjaGVzIG9uZSBtaWdodCB1c2UsIGFuZCB3ZSB3aWxsIGNvdmVyIHRoZSB0aGVvcnkgYmVoaW5kIGJvdGgKCgojIyBUaGVvcnkgUGFydCBJOiBPdmVyLXJlcHJlc2VudGF0aW9uIGFuYWx5c2lzCgotICJUaHJlc2hvbGQtYmFzZWQiOiByZXF1aXJlIGRlZmludGlvbiBvZiBhIHN0YXRpc3RpY2FsIHRocmVzaG9sZCB0byBkZWZpbmUgbGlzdCBvZiBnZW5lcyB0byB0ZXN0IChlLmcuIEZEUiA8IDAuMDEpCi0gSHlwZXJnZW9tZXRyaWMgdGVzdCBvciBGaXNoZXIncyBFeGFjdCB0ZXN0IGdlbmVyYWxseSB1c2VkLgoKVGhlIHF1ZXN0aW9uIHdlIGFyZSBhc2tpbmcgaGVyZSBpczsKCj4gKioqIkFyZSB0aGUgbnVtYmVyIG9mIERFIGdlbmVzIGFzc29jaWF0ZWQgd2l0aCBUaGVtZSBYIHNpZ25pZmljYW50bHkgZ3JlYXRlciB0aGFuIHdoYXQgd2UgbWlnaHQgZXhwZWN0IGJ5IGNoYW5jZSBhbG9uZT8iKioqCgpXaGVyZSBUaGVtZSBYIGNvdWxkIGJlIGdlbmVzIGJlbG9uZ2luZyB0byBhIHBhcnRpY3VsYXIgR08gKEdlbmUgT25vdG9sb2d5KSB0ZXJtIG9yIHBhdGh3YXkuCgpMZXQncyBpbWFnaW5lIHRoYXQgd2UgaGF2ZSBhIGJhZyBmdWxsIG9mIGJhbGxzLiBFYWNoIGJhbGxzIHJlcHJlc2VudHMgYSBnZW5lIGluIHRoZSAqZ2VuZSB1bml2ZXJzZSouIAotIFBhaW50IHRoZSBiYWxscyByZXByZXNlbnRpbmcgb3VyIHNlbGVjdGVkIGxpc3QgZ3JleSwgYW5kIHBhaW50IHRoZSByZXN0IHJlZC4KCgohW10oaW1hZ2VzL2JhZy1hbmQtYmFsbHMucG5nKQoKSW4gdGhpcyBzbWFsbCBleGFtcGxlLCB3ZSBjYW4ga25vdyBpbiBhZHZhbmNlIHRoZSB0b3RhbCBudW1iZXIgb2YgYmFsbHMgYW5kIHRvdGFsIG51bWJlciBpbiBvdXIgY2F0ZWdvcnkgb2YgaW50ZXJlc3QKCi0gVG90YWwgbnVtYmVyIG9mIGJhbGxzOiA0MAotIFRvdGFsIG51bWJlciBvZiBpbnRlcmVzdGluZyAoZ3JleSkgYmFsbHM6IDEwCgpOb3csIGxldHMgc2VsZWN0IGEgbnVtYmVyIChzYXksIDEyKSBvZiB0aGUgYmFsbHMgYXQgcmFuZG9tIHdpdGhvdXQgc2VlaW5nIGludG8gdGhlIGJhZyBhbmQgbG9vayBhdCB3aGF0IHdlIGdldAoKIVtdKGltYWdlcy9waWNrZWQtYmFsbHMucG5nKQoKCgpXZSBjYW4gZXhwcmVzcyBhIHBhcnRpY3VsYXIgY2hvaWNlIG9mIGJhbGxzIGFzIGEgdGFibGU6LQoKfCAgfCBJbiBTZWxlY3Rpb24gfCBOb3QgSW4gU2VsZWN0aW9uIHwgVG90YWwKLS0tLS0tLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0gfCAgLS0tLS0tLS0tLS0tLSB8IC0tLS0tLS0tLS0tLS0gfCAKR3JleSBTZWxlY3RlZCB8IDggfCAyIHwgMTAgfApHcmV5IE5vdCBTZWxlY3RlZCAgfCA0IHwgMjYgfCAzMCB8ClRvdGFsIHwgMTIgfCA2IHwgNDAgfAoKCgpUaGUgZm9ybXVsYSBmb3IgRmlzaGVycyBleGFjdCB0ZXN0IGlzOwoKJCQgcCA9IFxmcmFje1xiaW5vbXthICsgYn17YX1cYmlub217YyArZH17Y319e1xiaW5vbXtufXthICtjfX0gPSBcZnJhY3soYStiKSEoYytkKSEoYStjKSEoYitkKSF9e2EhYiFjIWQhbiF9ICQkCndpdGggOi0KCnwgIHwgSW4gU2VsZWN0aW9uIHwgTm90IEluIFNlbGVjdGlvbiB8IFRvdGFsCi0tLS0tLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tIHwgIC0tLS0tLS0tLS0tLS0gfCAtLS0tLS0tLS0tLS0tIHwgCkdyZXkgU2VsZWN0ZWQgfCBhIHwgYiB8IGEgICsgIGIgfApHcmV5IE5vdCBTZWxlY3RlZCAgfCBjIHwgZCB8IGMgKyBkIHwKVG90YWwgfCBhICsgYyB8IGIgK2QgfCBhICsgYiArIGMgKyBkICg9bikgfAoKCgoKb3IgbGVzcyBmb3JtYWxseTsKCipQID0gKHdheXMgb2YgY2hvb3NpbmcgZ3JleSBiYWxscykgWCAod2F5cyBvZiBub24tZ3JleSBiYWxscyBhbW9uZ3N0IHN1YnNldCkgLyB3YXlzIG9mIGNob29zaW5nIHN1YnNldCoKCgpGb3J0dW5hdGVseSwgd2UgaGF2ZSBzb2Z0d2FyZSB0byBjYWxjdWxhdGUgdGhlc2UgcXVhbnRpdGllcyEKCiMjIFNvZnR3YXJlIGZvciBjb25kdWN0aW5nIGEgb3Zlci1yZXByZXNlbnRhdGlvbiB0ZXN0IChnb3NlcSkKCldlIHdpbGwgYmUgdXNpbmcgW2Bnb3NlcWBdKGh0dHBzOi8vd3d3LmJpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS9iaW9jL2h0bWwvZ29zZXEuaHRtbCksIHdoaWNoIGlzIGEgc29mdHdhcmUgcGFja2FnZSBhdmFpbGFibGUgdGhyb3VnaCBbQmlvY29uZHVjdG9yXSh3d3cuYmlvY29uZHVjdG9yLm9yZy8pLiBIb3dldmVyLCByYXRoZXIgaGF2aW5nIHRvIHdyaXRlIFIgY29kZSwgd2Ugd2lsbCBiZSB1c2luZyB0aGUgcGFja2FnZSB0aHJvdWdoIG91ciBpbnN0aXR1dGUncyBHYWxheHkgc2VydmVyLgoKVGhpcyBwYWNrYWdlIGhhcyBiZWVuICpzcGVjaWZpY2FsbHktZGV2ZWxvcGVkKiBmb3IgdXNlIHdpdGggUk5BLXNlcSBkYXRhLiBQbGVudHkgb2YgbWV0aG9kcyBoYXZlIGJlZW4gYXBwbGllZCB0byBtaWNyb2FycmF5IGRhdGEsIGJ1dCB0aGUgYXNzdW1wdGlvbnMgbWlnaHQgbm90IGhvbGQgZm9yIFJOQS1zZXEuIEZvciBpbnN0YW5jZSwgYGdvc2VxYCB3aWxsIGFkanVzdCBmb3IgdGhlIGxlbmd0aCBvZiBhIGdlbmUuCgojIyBQcmVwYXJpbmcgdGhlIGRhdGEgZm9yIGFuIG92ZXItcmVwcmVzZW50YXRpb24gdGVzdAoKV2UgY2FuIHN0YXJ0IHdpdGggdGhlIGxpc3QgdGhhdCBjb250YWlucyByZXN1bHRzIGZvciAqYWxsKiBnZW5lczsgTGV0J3MgdGFrZSBgdDQ3ZF9UcmVhdG1lbnRfREVBX1Byb2ctdnMtQ29udHJvbF9hbGwuY3N2YAoKYGBge3J9CmRlVGFibGUgPC0gcmVhZC5jc3YoInQ0N2RfVHJlYXRtZW50X0RFQV9Qcm9nLXZzLUNvbnRyb2xfYWxsLmNzdiIpCmRlVGFibGUKYGBgCgpUaGUgZ29hbCBpcyB0byBvYnRhaW4gYSB0YWJsZSB3aXRoIHR3byBjb2x1bW5zOyB0aGUgZmlyc3QgY29udGFpbmluZyB0aGUgZ2VuZSBpZGVudGlmaWVyLCBhbmQgdGhlIHNlY29uZCBiZWluZyBgVFJVRWAgb3IgYEZBTFNFYCBpbmRpY2F0aW5nIHdoZXRoZXIgdGhlIGdlbmUgaXMgZGlmZmVyZW50aWFsbHktZXhwcmVzc2VkIGF0IGEgZ2l2ZW4gY3V0LW9mZiAoZS5nLiBgMC4wNWApLiAKClRoZXJlIGFyZSBzZXZlcmFsIHdheXMgb2YgZG9pbmcgdGhpcy4gSWYgeW91IHdpc2ggdG8gc2VlIHRoZSBSIGNvZGUgKHVzaW5nIGBkcGx5cmApLCBjbGljayB0aGUgJ0NvZGUnIGJ1dHRvbiBvbiB0aGUgcmlnaHQuCgpgYGB7ciBtZXNzYWdlPUZBTFNFfQpsaWJyYXJ5KGRwbHlyKQpkZVRhYmxlIDwtIHJlYWQuY3N2KCJ0NDdkX1RyZWF0bWVudF9ERUFfUHJvZy12cy1Db250cm9sX2FsbC5jc3YiKQptdXRhdGUoZGVUYWJsZSwgREUgPSBwYWRqIDwgMC4wNSkgJT4lIAogIG11dGF0ZShERSA9IGlmZWxzZShpcy5uYShERSksRkFMU0UsREUpKSAlPiUgCiAgc2VsZWN0KFgsIERFKSAtPiBmaWx0ZXJlZFRhYmxlCiAgd3JpdGUudGFibGUoZmlsdGVyZWRUYWJsZSwiZGUtdGFibGUtZm9yLWdvc2VxLnR4dCIscXVvdGU9RkFMU0Uscm93Lm5hbWVzPUZBTFNFKQpmaWx0ZXJlZFRhYmxlCmBgYAoKTWFuaXB1bGF0aW5nIHRoZSBmaWxlIGNhbiBhbHNvIGJlIGRvbmUgaW4gR2FsYXh5LgoKIyMgSW1wb3J0aW5nIHRoZSBHZW5lIGxpc3QgaW50byBHYWxheHkKCiMjIyBJbXBvcnQgdGhlIGNzdiBmaWxlCgoqKipHZXQgRGF0YSoqKiAtPiAqKipVcGxvYWQgRGF0YSoqKgoKIVtdKGltYWdlcy9nYWxheHktMS5wbmcpCgojIyMgQ29udmVydCB0byB0YWJ1bGFyIGZvcm0KCioqKlRleHQgTWFuaXB1bGF0aW9uKioqIC0+ICoqKkNvbnZlcnQgZGVsaW1pdGVycyB0byBUQUIqKioKClNlbGVjdCAqQ29tbWFzKiBmcm9tIGRyb3AtZG93biBtZW51CgohW10oaW1hZ2VzL2dhbGF4eS0yLnBuZykKCiMjIyBSZW1vdmUgaGVhZGVyIGxpbmUKCioqKlRleHQgTWFuaXB1bGF0aW9uKioqIC0+ICoqKlNlbGVjdCBsYXN0IGxpbmVzIGZyb20gYSBkYXRhc2V0KioqCgpNYWtlIG5vdGUgb2YgaG93IG1hbnkgbGluZXMgaW4gZmlsZSwgKk4qIGFuZCByZW1vdmUgKk4tMSogbGFzdCBsaW5lcyB3aXRoIHRoaXMgdG9vbAohW10oaW1hZ2VzL2dhbGF4eS0zLnBuZykKCiMjIyBSZXRhaW4gY29sdW1ucyBvZiBpbnRlcmVzdAoKUHV0ICpjMSxjMixjMyxjNCxjNSxjNixjNyxjMTIqIHRvIHJldGFpbiBjb2x1bW5zIDEsMiwzLDQsNSw2LDcgYW5kIDEyCgoqKipUZXh0IE1hbmlwdWxhdGlvbioqKiAtPiAqKipDdXQgY29sdW1ucyBmcm9tIGEgdGFibGUqKioKCiFbXShpbWFnZXMvZ2FsYXh5LTQucG5nKQoKIyMjIFRoZSBjbGVhbmVkIGRhdGFzZXQKClRoZSBvdXRwdXQgZnJvbSB0aGUgcHJldmlvdXMgc3RlcCB3aWxsIGJlIHJlZmVycmVkIHRvIGFzIG91ciAqY2xlYW5lZCBkYXRhc2V0KgoKIVtdKGltYWdlcy9nYWxheHktNS5wbmcpCgoKIyMgUGVyZm9ybWluZyBhIEdlbmUgU2V0IEFuYWx5c2lzIGluIEdhbGF4eQoKRnJvbSBvdXIgY2xlYW5lZCB0YWJsZSBpbiB0aGUgcHJldmlvdXMgc3RlcHMsIHdlIG5lZWQgdG8gY29tcHV0ZSBpZiBlYWNoIGdlbmUgaXMgREUgYXQgMC4wNSBzaWduaWZpY2FuY2UgbGV2ZWwKCiMjIyBBZGRpbmcgYW4gZXh0cmEgY29sdW1uCgoqKipUZXh0IE1hbmlwdWxhdGlvbioqKiAtPiAqKipDb21wdXRlIGFuIGV4cHJlc3Npb24gb24gZXZlcnkgcm93KioqCgpVc2UgdGhlIGV4cHJlc3Npb24gYGM3PDAuMDVgIHRvIHRlc3QgaWYgdGhlIGFkanVzdGVkIHAtdmFsdWUgaXMgbGVzcyB0aGFuIDAuMDUKCiFbXShpbWFnZXMvZ2FsYXh5LTYucG5nKQoKV2hpY2ggc2hvdWxkIGdpdmUgc29tZXRoaW5nIGxpa2UgdGhpcwoKIVtdKGltYWdlcy9nYWxheHktNy5wbmcpCgojIyMgRXh0cmFjdGluZyB0aGUgY29sdW1ucyBuZWVkZWQgZm9yIGdvc2VxCgoqKipUZXh0IE1hbmlwdWxhdGlvbioqKiAtPiAqKipDdXQgY29sdW1ucyBmcm9tIGEgdGFibGUqKioKQ2hvb3NlIGBjMSxjOWAgYXMgdGhlIGNvbHVtbnMgdG8gY3V0CgohW10oaW1hZ2VzL2dhbGF4eS04LnBuZykKCldoaWNoIHNob3VsZCBnaXZlIHRoZSBmb2xsb3dpbmcKCiFbXShpbWFnZXMvZ2FsYXh5LTkucG5nKQoKIyMjIFJ1bm5pbmcgZ29zZXEKCioqKk5HUzpSTkEtc2VxKiogLT4gZ29zZXEKClNlbGVjdCBHZW5vbWUgdmVyc2lvbiAqaGcxOSogaW4gdGhpcyBjYXNlLiBDb3VsZCBhbHNvIHNwZWNpZnkgd2hpY2ggdHlwZSBvZiBpZGVudGlmaWVycyBjYW4gYmUgZm91bmQgaW4gdGhlIGZpcnN0IGNvbHVtbi4gKkVuc2VtYmwqIElEIGlzIGNvcnJlY3QgaW4gdGhpcyBjYXNlLgoKIVtdKGltYWdlcy9nYWxheHktMTAucG5nKQoKCiMjIFRoZW9yeSBQYXJ0IElJOiBUaHJlc2hvbGQtZnJlZQoKRm9yIHRoZXNlIHRlc3RzLCB3ZSBkb24ndCBoYXZlIHRvIHNwZWNpZnkgYSBzdGF0aXN0aWNhbCB0aHJlc2hvbGQgYW5kIHVzZSB0aGUgdGVzdCBzdGF0aXN0aWNzIGZyb20gKmFsbCogZ2VuZXMgYXMgdGhlIGlucHV0IHRvIHRoZSB0ZXN0LiBUaGUgcG9wdWxhciAqR2VuZSBTZXQgRW5yaWNobWVudCBBbmFseXNpcyAoR1NFQSkqIHVzZXMgdGhpcyBhcHByb2FjaC4gVGhlc2UgdGVzdHMgY2FuIGJlIHVzZWQgdG8gZGV0ZWN0IGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGZvciBhIGdyb3VwIG9mIGdlbmVzLCBldmVuIHdoZW4gdGhlIGVmZmVjdHMgYXJlIHRvbyBzbWFsbCBvciB0aGVyZSBpcyB0b28gbGl0dGxlIGRhdGEgdG8gZGV0ZWN0IHRoZSBnZW5lcyBpbmRpdmlkdWFsbHkuCgohWypTdWJyYW1hbmlhbiBldCBhbCwgUE5BUyAyMDA1Kl0oaW1hZ2VzL0dTRUEucG5nKQoKLSBSYW5rIGFsbCB5b3VyIGdlbmVzIGFjY29yZGluZyB0byB0ZXN0IHN0YXRpc3RpYyBvciBmb2xkLWNoYW5nZQotIEZvciBhIHBhcnRpY3VsYXIgZ2VuZSBzZXQgb2YgaW50ZXJlc3QsIGxvb2sgZG93biB0aGUgZ2VuZSBsaXN0IGFuZCBjb21wdXRlIGEgc2NvcmUKICAgICsgR28gdXAgb25lIHVuaXQgb24gdGhlIHktYXhpcyBpZiB0aGUgZ2VuZSB5b3UgZW5jb3VudGVyIGlzIGNvbnRhaW5lZCBpbiB5b3VyIGdlbmUgbGlzdAogICAgKyBHbyBkb3duIG9uZSB1bml0IGlmIHRoZSBnZW5lIGlzIG5vdCBjb250YWluZWQgaW4gdGhlIGdlbmUgbGlzdAogICAgKyBQbG90IHRoZSBzY29yZSBhZ2FpbnN0IHBvc2l0aW9uIGluIGdlbmUgbGlzdAogICAgKyBJZiB0aGVyZSBpcyBhIHBlYWsgYXQgdGhlIGxlZnQgb3IgcmlnaHQsIHRoZW4geW91ciBzZXQgaXMgZW5yaWNoZWQuCgoKVGhlIEJyb2FkIGluc3RpdHV0ZSBwcm92aWRlcyBbYSB2ZXJzaW9uIG9mIEdTRUFdKGh0dHA6Ly9zb2Z0d2FyZS5icm9hZGluc3RpdHV0ZS5vcmcvZ3NlYS9pbmRleC5qc3ApIHRoYXQgY2FuIGJlIHJ1biB2aWEgYSBqYXZhIGFwcGxpY2F0aW9uLiAKVGhlIHJlcXVpcmVkIGlucHV0IHRleHQgY29udGFpbnMgYSBzZXQgb2YgZ2VuZSBpZGVudGlmaWVycyB0aGF0IGFyZSBvcmRlcmVkIGFjY29yZGluZyB0byBzb21lIHRlc3Qgc3RhdGlzdGljLgoKYGBge3J9CmxpYnJhcnkoZHBseXIpCmRlVGFibGUgPC0gcmVhZC5jc3YoImRlc2VxMl9yZXN1bHRzL3Q0N2RfVHJlYXRtZW50X0RFQV9Qcm9nLXZzLUNvbnRyb2xfYWxsLmNzdiIpCmRlVGFibGUgJT4lIGFycmFuZ2UoZGVzYyhzdGF0KSkgJT4lIAogIGRwbHlyOjo6c2VsZWN0KHN5bWJvbCwgc3RhdCkgJT4lIAogIGZpbHRlcighaXMubmEoc3ltYm9sKSkgLT4gb3JkZXJlZFRhYmxlCm9yZGVyZWRUYWJsZQpgYGAKCgoKIyMgUHJlcGFyaW5nIHRoZSBnZW5lIHNldCBmb3IgYSBHU0VBIGFuYWx5c2lzCgojIyMgUmFua2luZyB0aGUgZ2VuZXMKCkZyb20gdGhlICpjbGVhbmVkKiB0YWJsZSBjcmVhdGVkIGFib3ZlLCBzb3J0IG9uIGNvbHVtbiA1ICh0aGUgdGVzdCBzdGF0aXN0aWMpCgoqKipGaWx0ZXIgYW5kIFNvcnQqKiogLT4gKioqU29ydCBkYXRhIGluIGFzY2VuZGluZyBvciBkZXNjZW5kaW5nIG9yZGVyKioqCgohW10oaW1hZ2VzL2dhbGF4eS0xMC5wbmcpCgpHaXZpbmcgdGhlIG91dHB1dDotCgohW10oaW1hZ2VzL2dhbGF4eS0xMS5wbmcpCgojIyMgQ3V0dGluZyB0aGUgY29sdW1ucwoKRXh0cmFjdCBjb2x1bW5zIDggKGdlbmUgc3ltYm9sKSBmb2xsb3dlZCBieSB0ZXN0IHN0YXRpc3RpYzsgYGM4LGM1YAoKKioqVGV4dCBNYW5pcHVsYXRpb24qKiogLT4gKioqQ3V0IGNvbHVtbnMgZnJvbSBhIHRhYmxlKioqCgohW10oaW1hZ2VzL2dhbGF4eS0xMi5wbmcpCgoKIyMgUnVubmluZyB0aGUgR1NFQVByZXJhbmtlZCB0b29sCgpJbiBvdXIgR2FsYXh5IHNlcnZlciB3ZSBwcm92aWRlIGFuIGludGVyZmFjZSB0b2BmZ3NlYWAsIHdoaWNoIGlzIGEgQmlvY29uZHVjdG9yIHBhY2thZ2UgZm9yIHBlcmZvcm1pbmcgdGhlIEdTRUEgYWxnb3JpdGhtIG9uIGEgcHJlLXJhbmtlZCBnZW5lIGxpc3QuCgpXZSBjYW4gZ2l2ZSBpdCB0aGUgdGFibGUgd2UgaGF2ZSBnZW5lcmF0ZWQgaW4gdGhlIHByZXZpb3VzIHN0ZXAKCgpHaXZlcyBhbGwgcmVzdWx0cyByYW5rZWQgYWNjb3JkaW5nIHRvLi4KCi0gQ2FuIHdlIGlkZW50aWZ5IGdlbmUgc2V0cyB0aGF0IGFyZSBzaWduaWZpY2FudCBhdCAwLjA1PwotIFdoYXQgZ2VuZXMgYXJlIG92ZXItIGFuZCB1bmRlci1yZXByZXNlbnRlZD8KLSBDYW4geW91IHJ1biB0aGUgdG9vbCBvbiBhbm90aGVyIGRhdGFzZXQ/